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0.  Introduction 


It  is  frequently  the  case  that  service  facilities,  designed  to  accom- 
mocate  in  an  optimal  fashion  certain  demand,  become  inadequate  due  to  tin- 
anticipated  increase  in  the  arrival  intensity  of  customers.  This  is  often 
manifested  by  an  excessive  waiting  time  of  customers  and  sometimes  even 
saturation  of  the  systems.  To  avoid  these  undesirable  phenomena  the  service 
intensity  of  the  system  should  be  adapted  to  suit  the  circumstances.  If 
the  nature  of  change  in  the  arrival  Intensity  is  not  completely  known  the 
problem  is  twofold.  One  has  to  make  proper  Inference  from  observations  on 
the  system  about  the  possible  epochs  of  increase  in  the  arrival  intensity 
and  concurrently  to  make  optimal  decisions  on  the  proper  service  capacity. 

In  the  present  paper  we  study  this  problem  for  the  special  case  of  only  one 
possible  change  in  the  arrival  intensity,  which  may  occur  at  unknown  time 
point.  The  model  of  at  most  one  change  in  the  arrival  intensity  may  be  con- 
sidered as  an  over  simplification  of  the  real  problem.  However,  our  objec- 
tive is  to  present  the  problem  and  show  an  approach  for  obtaining  an  optimal 
solution.  He  allow  only  N possible  additions  to  the  initial  service 
capacity  of  the  system.  He  assume  that  one  can  only  Increase  the  capacity. 

We  further  restrict  attention  to  one-server  Markovian  queueing  systems. 

For  such  systems  we  develop  a procedure  for  the  optimal  adaptation  of  the 
capacity.  This  procedure  is  based  on  a cost  structure  which  consists  of 
the  operating  cost,  set-up  cost  and  holding  cost.  Furthermore,  the  optimi- 
zation is  carried  within  a Bayesian  framework.  We  assume  that  the  time  till  

the  change  in  the  arrival  Intensity  follows  an  exponential  prior  distribution.  Vl 

i 

The  Markovian  decision  procedure  is  accordingly  a function  of  two  state-  □ 


variables:  the  number  of  customers  in  the  system  and  the  posterior  proba- 


bility that  the  change  has  already  occurred.  The  optimality  criterion  is  j|>|[j[y  (jppfg 
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to  minimize  the  total  expected  discounted  cost  for  the  entire  future. 

Since  we  make  decisions  under  uncertainty,  there  is  the  option  of  changing 
the  service  capacity  in  N steps.  We  provide  here  an  algorithm  for  such 
adaptation.  In  Section  1 we  present  the  problem  in  more  specific  terms. 

In  Section  2 we  study  the  posterior  probabilities  process.  In  Section  3 we 
present  the  formulae  for  the  expected  cost  functions  at  the  maximal  capacity. 
On  the  basis  of  these  functions  we  develop  in  Section  4 the  general  func- 
tional equations  describing  the  minimal  expected  cost  functions,  under  the 
plausible  assumption  that  the  optimal  adaptation  is  a non-decreasing  func- 
tion of  each  one  of  the  state  variables.  This  assumption  is  not  essential 
but  helpful  in  simplifying  the  development.  Moreover,  in  Section  5 we  pro- 
vide a piece-wise  linear  approximation,  which  reduces  the  integral  equations 
previously  developed  to  simpler  difference  equations.  An  iterative  method 
for  solving  these  equations  is  discussed  in  Section  6.  In  this  section  we 
provide  also  computational  algorithm  which  combines  the  solution  of  the 
difference  equations  with  the  determination  of  the  optimal  adaptation  rules. 
The  results  of  the  present  study  can  be  generalized  to  cases  where  the 
adaptation  of  capacity  is  done  by  adding  servers  to  the  system. 

The  literature  on  the  optimal  control  of  queueing  systems  consists 
already  of  a few  hundred  papers.  Reviews  of  the  basic  problems  and 
approaches  can  be  found  in  Crabill,  Gross  and  Magazine  (1976),  Sobel  (1974) 
and  Stidham  and  Prabhu  (1974).  The  present  study  is  related  directly  only 
to  the  previous  studies  of  Zacks  and  Yadin  (1970)  and  Yadin  (1977).  These 
specific  papers  can  be  ascribed,  according  to  the  classification  of  Crabill, 
Gross  and  Magazine  (1976)  to  the  general  category  of  "service  process  control" 
of  dynamic  systems.  We  prefer,  however,  to  call  our  process  an  adaptation 
process,  since  the  service  capacity  cannot  be  decreased  but  only  increased. 
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1.  The  basic  problem 

Consider  a single  server  queueing  system  in  which  arrivals  follow  a 
Poisson  process.  This  process  starts  with  an  arrival  rate  \q  and  at  an 
unknown  time  point,  t the  arrival  rate  changes  to  A^.  We  assume  that 
\q  and  are  known  and  that  0 < \q  < < «. 

We  adopt  a Bayesian  model,  according  to  which  the  epoch  t of  shift 
from  Aq  to  A^  is  assumed  to  be  a random  variable  having  a prior 
exponential  distribution  with  mean  1/a.  The  service  times  of  this  system 
are  conditionally  Independent  random  variables  having  exponential  distri- 
butions. The  capacity  is  defined  as  the  expected  number  of  customers  that 
the  system  can  serve  per  unit  of  time.  The  system  is  set  Initially  to 
have  a capacity  Uq,  which  is  related  to  the  initial  arrival  rate  A^. 

It  is  possible,  however,  to  increase  the  capacity  in  N steps  from  y^ 
to  iijgt  according  to  specified  values: 

"O  <»1<"‘<  % • 

The  time  points  at  which  the  changes  in  capacity  take  place  are  called 
adaptation  points.  There  is  no  possibility  of  decreasing  the  capacity 
of  the  system.  The  problem  is  to  determine  the  optimal  adaptation  times 
in  a way  which  minimizes  the  total  expected  discounted  cost  of  operating 
the  system.  This  cost  consists  of  three  elements:  the  cost  of  maintain- 
ing service  at  a given  capacity  per  unit  of  time,  S(y);  the  cost  of 
holding  x customers  at  the  system  one  unit  of  time,  Q(x)  and  the  cost 
of  increasing  the  capacity  (set-up  cost)  from  y^  to  y^  (Os  i<  j £ K) , 
c(i,j). 


-4- 


2.  Posterior  distributions  based  on  the  arrival  times. 

Let  A(t)  be  the  arrival  rate  at  time  t,  0 < t < «.  According  to 
the  model 


(2.1) 


A(t)  = \QI  {t  < t]  + \±I  £t  2 t}; 


where  I [A]  is  the  indicator  function  of  t,  with  respect  to  the  set  A. 
Accordingly,  h(t)  is  a random  function.  We  derive  here  the  posterior 
distributions  of  h(t),  given  the  past  history  of  the  arrival  process. 

Let  e1  denote  the  length  of  time  elapsed  since  the  most  recent  arrival 
till  the  present  time  t.  More  generally,  counting  the  arrivals  from  the 
time  t in  a backward  fashion,  let  0^  (k  = 1,2,...)  denote  the  length  of 
time  since  the  k-th  arrival  till  the  present  time  t.  In  this  notation, 
the  last  k arrival  time  points,  prior  to  t are  t -.0^, 
t - 02,  ..,t  - 0^.  We  also  set  0^  h 0.  Let  N(t)  denote  the  number  of 
arrivals  in  the  time  interval  (0,t].  In  the  event  of  [N(t)  = n]  we  set 
0n+l  = and  the  posterior  probability  of  (A(t)  = \Q},  given  §t  = (N(t)  = n. 


(2.2) 


aJ 

p (ACt)  _ >-0|st}  - — * 

a J e 
0 


e"ct/rL(T  ,§t)dr 


l(t  >St)dT 


where  L(TjSt)  is  the  likelihood  function  of  t for  a given  (t,§t).  This 
likelihood  function  is  expressed  by 


(2.3) 


■ n 

L(Ti»t)=^e  0 I{T  > t) 

f . n-k.  k _X0T-Xl(T-t) 
+ L>  K K e 

k=0  u 1 


H'-Vl  * T * ‘-V 
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Substituting  (2.3)  in  (2.2)  we  obtain  after  proper  integrations  the  following 
formula  for  the  posterior  probabilities 


(2.l>)  P{A(t)  = \0|8t)  ■ 


X1  ' \>  ' a 


W“ 


2.  kf  -®klWa)  *‘WWlJ' 


*«s  »*[ 

k=0  L 


- e 


] 


where  p = X^/X^.  Define  the  stochasic  process  (A(t),t  a 0}  which,  at 
every  point  of  time  t,  is  a function  of  the  sufficient  statistic  g^,  as 
given  by  the  formula 


(2.5) 


where  T]  = (X^-X^-a)/a.  Notice  that  A(0)  = 0 with  probability  one.  Every 
decision  process  based  on  the  behavior  of  the  relization  of  the  posterior 
probabilities  process  = X^ [ g^}  can  be  equivalently  based  on  the 

realization  of  the  A(t)-process,  since 

<2-6>  f{A«0  - - vrfcT  " 0 £ ‘ < - 


We  investigate  here  some  of  the  properties  of  the  A(t)-process.  We  focus 
attention  on  the  case  of  T)  > 0.  The  case  of  T)  fi  0 is  not  of  interest, 

since  it  implies  an  early  change  of  XQ.  The  process  A(t)  is  induced  by 
the  non-homogeneous  Poisson  process  in  the  following  fashion.  Consider  the 

time  interval  [t,t  + h].  If  there  is  no  arrival  at  this  time  interval  then 
(2.6)  A(t  + h)  = e"a:T1h[A(t)  -1]  + 1. 


On  the  other  hand,  if  t is  an  arrival  point  then 


(2.7) 


A(t+)  = pA(t-). 
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In  other  words,  the  A(t)  process  has  discontinuities  at  the  arrival  points. 
At  t-points  which  are  not  arrival  points  the  process  has  a smooth  realization 
with  derivatives  given  by 


(2.8) 


A(t)  = -aT)[A(t)  - 1]. 


This  shows  that  A(t)  is  an  increasing  process  as  long  as  A(t)  < 1.  Once 
A(t)  jumps  above  the  level  A = 1 it  always  stays  there.  Moreover,  after 
crossing  the  level  A = 1 the  process  is  strictly  decreasing  between  arrival 
points,  at  which  it  jumps  upwards. 

Let  E^(a),  i = 0,1,  designate  the  conditional  expectation  of  A(t) 
immediately  after  an  arrival  point,  given  that  the  level  of  the  process 
immediately  after  the  previous  arrival  is  a,  and  A(t)  = \.  for  all  t 
between  these  two  arrival  points.  This  conditional  expectation  is 


(2.9) 


OO  -\,t 


Ei(a)  = e 1 |^e"a^t[a  - l]  + lj  dt 


= (Xia4txTl)  ’ 1 = °>1- 

Since  we  consider  here  cases  of  ^ > 0 one  could  immediately  verify  that 
E^(a)  > a,  i = 0,1.  Thus,  the  imbedded  sequence  of  A(t+)  at  the  arrival 
points  is  a submartingale  which  diverges  a.s.  to  infinity. 

This  in  conjunction  with  (2.6)  implies  that  the  posterior  probability 
P{A(t)  = \)l®t^  "*  ^ a,s,>  as  t -»  oo,  even  if  there  is  no  change  in 

This  is  not  surprising,  since  the  prior  probability  P[A(t)  = ■*  0 

exponentially  fast,  as  t -♦  oo. 


L,,« 1 IIIM.I.II'I  w _UW*1|..IIM*_  I , lull  Iifininjiwii  .1  I , I p!nii.iuiiiM  _ ___  _ 
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3.  The  Expected  Cost  Functions  at  Maximal  Service  Capacity. 

Due  to  Che  Markovian  scruccure  of  Che  queueing  process,  Che  Cocal 
expected  discounted  cost  for  the  entire  future,  at  epoch  t,  given  that 
A(t)  - A^  (i  ■ 0,1);  the  system  operates  at  capacity  (j  * 1,...,N) 

and  there  are  X(t)  * x customers  in  the  queue  is  independent  of  t. 

We  denote  this  cost  function,  therefore,  by  (x) . In  the  present  sec- 


tion we  present  the  expressions  for  M^Cx)  and  Mqn(x). 


3 . 1 Derivation  of  M, ..(x)  . 


Let  jij(x)  ■ Pj  if  xi  1 and  Pj(0)  = 0.  The  change  in  the  queue 
size,  x,  is  induced  by  two  independent  Poisson  processes  of  rates  A^ 
and  Pj(x).  Accordingly,  the  time  until  the  next  change  in  x is  expo- 
nentially distributed,  with  parameter  A^  + p^(x).  At  the  time  of  change 
the  queue  increases  from  x to  x + 1 with  probability  A^/(A^  + p^(x)) 

and  decreases  to  x - 1 with  probability  p.(x)/(A  + p.(x)).  Thus, 

J 1 J 

M^(x)  satisfies  the  functional  equation 


(3.1)  mln(x)  = J + HKW)  * exp{-(A1  + pN(x))e) 

0 

• | f [ss ) * «<*>]  * * '"p9  [ rV_  v**1) 

[o  L\|>NW 


nN(x) 
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Notice  that  e ^ , 0 < p < «,  is  the  present  worth  factor.  Let 


(3.2) 


H(n,x)  = J IS(n)  + Q(x)]. 


Notice  that  H(y,,x)  is  the  total  discounted  cost  of  maintaining  a service 
rate  ^ and  queue  size  x for  the  entire  future.  Substituting  H(p,,x) 
in  (3.1)  we  obtain  the  functional  equation 


(3.3) 


M3N(x)  = 


X1M1T/X+1)'^N(x)M1n(X~1)  * pH(Vx) 
>sWx)+P 


The  solution  of  this  functional  equation  can  be  approximated  for  any  H(u,x) 
function.  However,  in  special  cases  one  could  get  the  solution  in  a closed 
form.  Zacks  and  Yadin  (1970)  derived  a closed  form  expression  for 
for  the  case  of  H(y,x)  = [S(y)  + qx].  The  explicit  solution  obtained  is 


(3.4) 


where 


(3.5) 


min(x)  *?  s(u 


N,+q 


* 2^  [X1  + + P ■ <<xi  + Ui+  P)2  ‘ «Xi'i>1/3  ' 


i * 0,1, ; j ■ 1, . . . ,N. 

3 . 2 Derivation  of  Mq^(x) . 

The  functional  equation  for  Mqjj(x)  is  derived  as  in  the  previous  case 
of  M^jj(x),  taking  into  account  that  the  arrival  rate  may  change  to 


before  the  change  in  the  queue  size.  Accordingly,  we  obtain  the  equation 


(3.6) 


where 


: x x0M0N^X+1^N^x^M0N(x-1^  + ^a+P^H  W,X^ 

= >^0+yN(x)+a+p 


-9- 


(3.7) 


A*N'X)  = Sfe  MlN(x)+pH(%'x)]‘ 


For  the  special  case  of  H(p,x)  = ^ [S(n)+qx]  we  obtain,  according  to 
(3.5)  and  (3.7) 


(3.8) 


- 5 K>  + ^55r!l * p 1 * 


x+1 

«1I  <»> 


p(a+p)U-l1N(P)) 

* 

The  function  H (^,x)  can  be  written  as  the  sum  of  the  two  functions 

(3.9)  H(l)(vx)  = i [s(nN)  + ^ q ^ +- J + qx^] 

and 


(3.10) 


H(2),  ^ _ 3 . a . sn/p^  , .x  M 

H (uN,x)  P a + p 1 - ^(p)  Wp)* 


f 1 

By  substituting  the  function  Hv  '(nN,x),  for  H (nN,x)  in  (3.6),  we 
obtain  a functional  equation  similar  to  (3.3).  The  solution  of  this 
equation  is  given  therefore,  in  analogy  with  (3.4),  by 


(3.11) 


w(l) 

“on 


U)  = p [S(pN}  + q ( 


\*  - n 


x + 


N 


C(a+p) 

1 - 50N(a+p) 


)]• 


where 


(3.12) 


* ct\1  + vkQ 
a + p 


/ o \ 

When  H'  '(uN,x)  is  substituted  in  (3.6)  the  solution  assumes  the 


form 
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(3.13) 


Vp) 


IS  W - f • 1 - 5„(p) 


§ 

“ 


^(P) 

qQ^(p)  ~ 1 - ~Wa+p)J 

P VVp)_Va+p)][Wa+pMlN(p)] 


^;vp> 

- Va+p) 


where  Gx(z, s),  |z|  £ 1,  0 < s < ® is  the  double  transform  of  the 

transition  distribution  of  the  M/M/l  queueing  process  with  Xq  and 

UN-  Mo-eover,  tQN(a+p)  = ^N/\)50N(a+p).  The  denominator  of  (3.17) 

(2) 

can  be  further  simplified  and  the  formula  of  (x)  becomes 


(3.14) 


where 


(3.15) 


§XH 

_ q R 

" p 1 - 


1(p)  §0N1(a+P^ 

T^TpT  1 - ^oN^a+p^ 


a ^(p) 


O'  IN 


= a/ [a  + (XQ  - X1)(l  - S^p))]. 

Finally,  since  equation  (3.6)  is  linear  its  solution  of  (3.10)  is 
the  sum  of  MqJ^(x)  and  M^(x).  Accordingly, 


(3.16) 


r x* . 

mqN(x)  = i s(pn}  + q x + — r1 


. . sSW  , _ ^(p> 

1 - §0N(q!+p)  1 “ 51N(P 


In  the  special  case  of  XQ  = X^  we  have  B = 1 and  (3.16)  is  reduced 
to  (3.4). 


In  the  present  section  we  provide  the  recursive  equations  on  which 
the  optimization  process  is  based.  The  decision  concerning  the  optimal 
service  capacity  depends  on  the  current  capacity,  pn  (n  = 0,1,  2,  . . .,N); 
on  the  number  of  customers  in  the  station,  x,  and  on  the  posterior 
probability  II(t)  = P{A(t)  = XjJS^). 

We  start  with  the  supposition  that  the  adaptation  policy  should  be 
monotone  in  the  queue  size,  X(t),  for  a fixed  value  of  the  posterior 
probability  II(t)  and  monotone  in  II(t)  for  a fixed  X(t).  In  other 
words,  if  for  a given  value  of  II(t)  it  is  optimal,  at  time  t,  to 
shift  from  p_  to  p.  (j  > n);  the  optimal  decision  for  all  x > X(t) 

J 

is  to  shift  from  u to  p (k  ^ j > n).  Similarly,  for  a given  value 
of  X(t),  and  since  n(t)  is  an  increasing  function  of  A(t),  an  opti- 
mal shift  from  p to  p (j  > n)  under  A(t)  implies  that  the  optimal 

n j 

shift  for  all  a > A(t)  is  from  p to  p (k  Ss  j > n).  We  further 

impose  the  restriction  that  no  change  in  the  service  capacity  will  take 

place  as  long  as  A(t)  < 1.  It  can  then  be  shown  that  under  these  condi- 
tions the  optimal  decision  epochs  are  those  at  which  arrivals  occur.  Due 
to  the  Markovian  properties  of  the  decision  process,  increase  in  capacity 
should  take  place  either  when  the  queue  size  increases  or  when  the  pos- 
terior probability  of  increases.  Since  the  posterior  probability  of 
X^  is  an  increasing  function  of  A(t)  and,  when  A(t)  i 1,  it  increases 
only  at  the  arrival  epochs.  The  epochs  of  increase  in  the  queue  size  and 
in  the  posterior  probability  coincide. 

Let  Kq(x, a)  be  the  minimal  (conditional)  expected  discounted  cost  at 
some  epoch  t,  given  that  X(t)  = x,  A(t)  = a and  the  capacity  is  pQ. 
Furthermore,  let  K,  (x, a)  (i  = 0,1)  denote  the  conditional  expected 
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discounted  cost,  given  p^,  X(t)  = x,  A(t)  = a and  /v(t)  = \ asso- 
ciated with  the  optimal  policy.  For  each  n,  and  a 


(fc.l) 


KQ(x,a)  = (aK.n(x,a)  + a) )/(?!  + a). 


Let  I^Cx,  a)  denote  an  indicator  function,  which  assumes  the  value  1 if, 

under  the  optimal  policy,  an  increase  from  p to  p.  (j  = n, ...,N) 

n j 

should  take  place  at  epoch  t for  which  X(t)  = x and  A(t)  = a;  and 

N 

assumes  the  value  zero  otherwise  2 I .(x,a)  = 1.  With  the  aid  of  the 

J-n  Dj 

indicator  functions  Inj(x,a)  we  determine  simultaneously  with  KinU’a) 

associated  functions  M,  (x,a)  (i  = 0,1)  satisfying  the  equations: 

in 


(*.2) 


N 


Kin(x#a)  = Sln1(x,a)[c(n,j)  +M11(x,a)], 


M 


i = 0,1;  n = 1,  ...,N  and 


(^.3) 


p H(p.x) 


+ p (x)  + p + ~a  + 
0 n 


• -(X0+pn(x)+i»-a)T 

J e 
0 


[X0K0,n^X+1,pe"°nlT^a"1+  p)  + 


Pn(x)M0^n(x-l,e  °^T(a-l)  + l)  + 
a n(x,e_Q^T(a  - l)  + l)  J dr  , 


and 


(fc.4) 


p H(un,x) 

^ + »„(*)  + ~P  + 

• -(X  +p  (x)+P)t  r -oTIt,  . , 

J e 1 n I ^iK!, nX+l’ Pe  (a  - l)  + p)  + 

0 L 

+ pn(x)Mln(x-l,e  °^T(  a - l)  + l)"J  dT  . 


-13- 


Having  determined  the  functions  M^n(x,  a),  we  consider  their  expectation 

(4.5)  Mq(x,  a)  = (a^a)  + T\HQn(x,a.))/(T\  + a). 

Equations  (4.1),  (4.2)  and  (4.5)  imply 

N 

(4.6)  K (x,a)  = Jl  (x,a)[c(n,j)  + M (x,a)]. 

j=n  riJ  J 

It  follows  that 

(4.7)  K (x,  a)  = min  (c(n,j)  + M (x,a)}. 

We  remark  that  for  n = N 


(4.8)  Kj^a)  = MjjUja)  = ( aM1I[(x)  + TlM0N(x))/(a  + 'll). 

Equations  (4.3)  and  (U.k)  can  be  expressed  in  terms  of  the  M-functions 
only,  by  substituting  the  K- functions  according  to  (4.2).  Accordingly, 


(4.9)  Min(x,a)  = H(nn,x) 

in 

qd  — ^ (x)  T i N 

+ 0 6 ln  ] i j Xn j ^x+1>  p(a-l)e"a71x  + p)  jc(n,j)  + 

+ M^fx+l,  pfa-lje-0^  + p)J  + nn(x)Min(x-l,  (a^l)e~°^T  + l) 


+ aMLn(x,  (a-lje-0^  + l) 


dr; 


where  wlQ(x)  = + un(x)  + a + p (i  = 0,l).  Notice  that  in  (4.9),  t 
designates  the  random  time  of  change  in  the  queue  size;  capacity  adapta- 
tion is  done  only  at  epochs  of  increase  in  the  queue  size  and  A(t)  = a 
changes  according  to  the  formulae  given  in  Section  2.  Making  the  trans- 
formation u = (aF.l)e_a^T  + 1,  (4.P)  is  reduced  to 


(4.10)  Min(x,a)  =r^yH(Vx)  + 

In 


-vln(x)/oti  «ln(*)/on-ir  » 


J <U-1) 

1 


x.  Si  (x+i,  pU) 
j=n  nj 


Cc(n»d)  + M^^Cx+l,  pu)]  + nn(x)Min(x-l,  u)  + 


aM^n  (x,u)J du  . 

Partition  the  interval  (l, a)  to  the  sets 


(4.11) 


®hj(x)  = (u;  I (x,pu)  =1),  j = n,  ...,N. 


Since  we  consider  only  monotone  policies,  these  sets  are  subintervals. 
The  functional  equations  are  then,  for  each  i = 0,1; 


(U.12)  MinU,.)  ♦ 

^35 S I (u-i) 

11  j=n  (**1) 


win(x)/aTVl 


X^c  (n,  j )+M^  (x+1,  pu)  ] 


+ ^n(x)Min(x-l,  u)  + aMLn(x,u)>du 


We  introduce  now  boundary  functions  for  optimal  policies,  a .(x), 

nj 

j = n, ...,N,  x = 0,1,...  such  that,  if  the  value  of  A(t)  lies  between 

a .(x)  and  a - (x)  it  is  optimal  to  increase  the  capacity  from  u 
nj  n,  j+x  n 

to  n..  We  formally  define  a (x)  = p and  a (x)  = ®,  for  each 
j nn  n,  N **  x 

n = 1, . . . , N and  x = 0, 1, . . . . Hie  connection  between  the  subintervals 


-15- 


Notice  that  for  all  j such  that  a < — : 
is  empty.  Accordingly,  define  the  index 


the  subinterval  © . (x) 

nj 


(4.l4) 


f a . (x) 

jQ(x,a)  = max ni  j,  a * — ^ — 


and  set  (j%4)+1(x)  = pa. 


5.  Piecewise  Linear  Approximation 

In  the  present  section  we  derive  a linear  functional  equation  which 
approximates  (4.12)  on  a denumerable  set  of  points.  More  specifically, 
consider  a partition  of  the  range  of  a according  to  the  logarithmic 

l/  A 

scale  r * p ' , where  £ is  a predetermined  integer.  Furthermore, 
consider  sequences  of  integer  boundary  points,  d^ (x),  in  the  new  scale 


such  that 


(5.1) 


d (x) 

a^U)  = r nJ  , j st  n. 


For  i = 0,1,  let 


(5-2) 


Gin(x»a)  = win(x)  {Xi  Kin(x+1*Pa)  + un(x)Min(x-l,a)  + a Mln(x,a)J  , 


In  these  terms,  functions  M^n(x, a)  at  the  points  a = r , k = 1,2,.. 
are  given  by 


(5.3) 


Min(x'rk)  = H(Vx)  + 


w (x) 

I - 1 (x,u)au 

001  V=1  v-1 

r 


For  each  v = 1, . . . , k,  define 


(5*1*)  s.  (x, v)  = — iSj 


''.Wr 


win(x) 


J (u-1)  aT 

rv”1 


G,  (x,u)du. 


and  write 


(5.5) 


Mi„<x'rk)  H<vx)  + 


. -w  (x)/dfll  k 
(rk  - 1)  111  I 


L S (x,v). 
v=l 


By  simple  substitution  we  can  verify  that  the  functions  Min(x, r ) 
can  be  determined  recursively,  for  each  k i 1,  according  to 


(5.6) 


Min(x»r  ) * 1 - 


r*-1  - 1 ''in1*'/01'  . 


k 

r - 1 


/ k-1  \ w,  (x)/an  k-1 

*^dt(77r)  r 


+ (r  - 1) 


S^n(x,  k). 


where,  for  k = 0, 


(5-7) 


Min(x'l}  =^7  |P  H(VX)  + 

\i  Kin(x+l,p)  + nn(x)Min(x-l,l)  + a ^(x,!) 


This  expression  is  obtained  directly  from  (4.12)  by  applying  the  L’Hospital 
rule.  We  approximate  now  the  functions  M^(x,  a)  piecewise  linearly  on 
the  intervals  (rv_\rv)  v = 1, 2, ...,k.  Hie  value  of  l which  deter- 
mines the  logarithmic  scale  determines  the  accuracy  of  the  approximation. 
The  functions  G^j(x, a)  are,  as  in  (5.2),  linear  combinations  of  M^n(x>a) 


I 

I 
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functions  within  these  intervals.  Thus,  they  can  be  represented  within 
each  interval  [rv"^,rV],  v = 0,1,...  by  the  linear  functions 


(5.8) 


Gln(x'a)  = Ai,Vn(x)  + *4n)(x)a  » 


v-1  ^ ^ v 

r s a s r , 


where 


(5-9)  - ^T-Vl  - <1,<*.rV‘1>]  • 

r - r i — 

If  we  substitute  G.  (x,  a)  in  (5«k)  we  obtain  the  approximation 


(5-10) 


S.n(x,k)  = (r  - 1) 
- 1) 


k ^Win(x)/«V*  k^ 


Gin(x'r  ) 


k-1  ^win(x)/^  * 


b(^) ( v\  <*!) 

Bin  win(x)  + al] 


0In<*»rk'1>  , 

I-  »ln(*) 


(r  - 1) 


aT) 


+ 1 


Wiaw 


- (r11-1  - 


] 


Substituting  (5*9)  and  (5*10)  in  (5*6)  we  obtain  a recursive  function 

for  the  approximating  function  M*n(x,rk),  k * 1,  where  M*n(x,l)  = Min(x>l)- 
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6.  Computing  Algorithm  For  Recursive  Solution 

We  provide  a computing  algorithm  for  the  solution  of  the  functional 
* / k\ 

equations  for  M^n(x, r ).  The  approach  is  to  determine  first  a reasonable 
initial  solution  and  improve  it  by  a sequence  of  iterations.  It  is 
desirable  here  to  express  the  equations  for  i = 0 and  i = 1 separately, 
since  these  equations  are  derived  from  two  separate  sources  (4.3)  and 
(4.4),  respectively.  They  were  combined  to  one  expression  in  (4.9) 
for  the  sake  of  uniform  treatment.  Following  the  general  backward 
induction  scheme  of  dynamic  programming  we  start  with  the  functions 
corresponding  to  We  compute  first  M^(o)  according  to  (3.4)  and 

MqN(0)  according  to  (3.1 6).  Then,  for  every  x = 1, 2, ...  we  compute 


min(x)  = + p (i  - 5^j(p)), 

(6.1)  * 

*W(x)  = *WX“1>  + p (j*  - ((l-B)SjN(a+p)  + B§^(p)J 

Ic 

and  by  (4.8)  we  determine  their  expected  value  M^(x,r  ).  For  n * N-l,  we 
start  with  the  initial  solution, 

= 1 

Ki,°N-l(x’rk)  3 c(N-1,N)  + Mi]f(x) 


(6.2) 
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for  x = 1,2, , k = 1,1+1,...  . This  initial  solution  corresponds  to 

the  case  in  which  one  increases  the  capacity  following  the  first  arrival. 

For  this  reason  we  set  d^|  N(x)  = 4*  Clearly,  this  is  not  necessarily 

an  optimal  policy.  The  following  iterations  are  designed  to  improve  the 

policy  and  simultaneously  to  adjust  the  expected  cost  functions.  Let 

mJV2  i(*,rk  ) denote  the  solutions  at  the  v-th  iteration;  v = 1, 2, ...  . 

1|  W*  -L 

Ihese  functions  are  determined  for  k = 1 recursively  according  to  the 
formulae 


(6.3) 


K Kw(i'r£)  + > 

mo,n-i^0,:l^  = \Q+a+p  ' *0  K0,N-l^1,r  ^ + qmi,n-i^0,:l^  + s^nti^|  ’ 

'QllW  - Kw(x+1>r£)  + 


hl-1  Ml, N-l(x”1# X)  + P H(^N-l'x)j 

“o.N-l^1^  = X0+‘iN_1+a+P  X°  K0,N-l^X+1,r  ^ + 

Vi  + a "i%iM  + p H(Vi'x)  ' • 

To  obtain  m|V^  ^(x, rk)  for  k a 1,  we  determine  first 

Gi,l- i(0'rk)  = Ki^i(r»rk+i) 

(6-u)  <5o(v»-i(°>rl')  - j,  * a4;iito'rk)  1 


1 


r(v)  rk\ 1 , \ (v-l)  (x+1  rk+^)  + 

gi,n-i  x> r ' ~ x‘1+^n.1+p  i ki,n-iu  1,r  ' 

" 

^N-l  M1  N-l^X“1,r  ^ ' ' 

p(v)  (x  rk}  _ 1 \ K^^^^fx+l  rk+^)  + 

G0,  N- 1 X* r } " V^N-l+P+a  *0  K0,N-1U  ’ 

>Vl  Mo“B-l(x-1’rk)  * aMS-l(x’rk,l  « 


For  the  sake  of  convenience,  define 


(6.5) 


wi,»-l(x)  = 


*■0  * >W(x)  * a * p ' 


if  i « 0 


li  * Vi(x)  * p > lf  1 - 1 > 


(6.6) 


r k , 1”I,h-i(x)/oT| 

.*  , kk  r - 1 

Ai,»-i(x-r  > s — 

L r - i _|  — j 

3*  (x  rk)  - 1 dll  ■ x _ A*  (x  rk)  ..r*"  — ~ I 

i'N‘1  ' ^...(x)  + ^ l ’ rk  - r^J 


The  functions  (x, rk)  are  given  then  by 

1 j W-  x 


(6‘7)  MM-l(x'rk)  ' AI,N-l<X'rK)Hi,VM-l(x'r’'’'t) 

* <A  - Al,B-l(x'rk))  7"^  HVl'*)  * 

£-  AM-1(,’rk)  - BI,».i<x'rk2]0S-i(x’rk'l) 

* BI,K-l(x’rk,0^»-l<X'rk)- 

Following  these  computations  we  determine  at  the  v-th  iteration  the 


kw(v) 


posterior  expectation 
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(6.8)  ^ [ri  n'^1(x,rt)  . rk  . 

Furthermore,  we  correct  the  K functions  and  determine  the  boundary- 
points  for  the  present  iteration.  Thus,  we  set 


(6.9) 


K^(x,rk)  = min  | *^^(x,rk),^^  (x,rk)} 

<£>,„<*)  = min  {ki4-i(x,rk^ < t4-i^x,rk^}  * 


Let 


(6.10) 


„(v)  / kv  _ 

Ki,N-l(x,r  } " 


■ if  k 2 


For  a given  k{V^](x, rk)  the  functions  1(x,  rk)  are  determined 

exactly  by  the  recursive  equations  developed  above.  At  the  final  stage 
of  the  v-th  iteration  the  function  (x, rk)  is  changed  to 

1(x,rk)  which  is  the  basis  for  the  (v  + l)st  iteration.  For  each 
x and  k,  the  sequence  {KN-i(x,rk);V  2 °)  is  non- increasing  and 
therefore  converges  to  a limit  as  v -*  ».  Moreover,  the  corresponding 


sequence  {4-1,h(x)*v  2 °)  is  non- decreasing.  This  is  verified  by 
observing  that  for  each  x,  if  k s d^V|  N(x)  the11  K^^^(x,rk)  = 
K^(x,rk).  Thus,  if  <4-l,N^  < 4-1^ ^ then  for  411  k» 

4rl,N(x)  K k KH-l^X,rk^  = KN-l^X,I'k^  0n  the  other  hand’ 

at  the  (v-l)st  iteration  with  k'  = ^“^(x),  (x, rk  ) < K^°j(x, rk  ). 

But  this  contradicts  the  fact  that  K^V|(x, rk)  £ K^V^"\x,rk)  at  every 
x and  k.  Under  the  following  condition  on  the  cost  functions 


(6.11) 


p C(N-1,N)  + (SG^)  - S(nN))  + X - nN)  < 0 


I 

I 
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there  exists  a sufficiently  large  value  of  x,  x say,  so  that  for  all 

x a x it  is  optimal  to  switch  to  even  if  A(t)  = XQ,  for  all  t. 

We  therefore  restrict  our  computations  only  for  x s x.  Similarly,  we 

restrict  the  computations  for  k £ k;  where  k is  sufficiently  large  so 

that  the  posterior  probability  that  A(t)  = X^  is  at  least  1 - e 

(e  > 0 arbitrarily  small)  for  all  k > k.  On  the  restricted  domain  of 

x s x and  k £ k,  the  iterations  converge  in  a finite  number  of  steps. 

For  n < N-l  the  procedure  is  very  similar.  However,  to  determine 

K^°^(x, rk)  we  set 
i,n  7 

K^(x,rk)  = min  (c(n,j)  + M.(x,  rk)  ) , 
n j>n  3 

(6.12)  d^(x)  = min  |k;K^°^  (x,rk)  = c(n,  j)  + M^(x,rk)}  # <3  = n, . . 


Vx'k)  = 


1 < lf  ilS)W5t<1?j(iw 


0 , if  otherwise 


, j =n, ...,N 


(6.13)  K^(x,rk)  = "L  I^(x,k)[c(n,  j)  + M.  . (x,  rk)  ] . 

1 j=n+l  J 3 

The  functions  M^j(x,  rk)  and  M^.(x,  r ) are  the  ones  obtained  by  the 
iterations  in  the  previous  stages  (j  > m).  The  rest  of  the  computations 
are  similar.  In  formula  (6.9)  we  should  replace  d^Vj  jj(x)  ^ 

(6.14)  d^V\x)  = min^k;K^(x,rk)  <M^(x,rk)J 


- - Kr1,w-anv,«) 


9 J = • • • 9 N 
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